{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Google Maps Review Summarizer - Day 2 Solution\n",
    "\n",
    "This notebook demonstrates web scraping Google Maps reviews and analyzing them with Llama 3.2 via Ollama.\n",
    "\n",
    "## Features:\n",
    "- Web scraping with Playwright\n",
    "- Async data extraction\n",
    "- LLM-based sentiment analysis\n",
    "- Local LLM (no API costs!)\n",
    "\n",
    "**Author:** İrem Bezci (r00trose)  \n",
    "**Course:** LLM Engineering - Week 1, Day 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup & Installation\n",
    "\n",
    "First, make sure you have the required packages installed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install required packages (run once)\n",
    "!pip install playwright ollama\n",
    "!playwright install chromium"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio\n",
    "import ollama\n",
    "from playwright.async_api import async_playwright\n",
    "from IPython.display import Markdown, display"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Functions\n",
    "\n",
    "### 1. Scroll Reviews Panel\n",
    "This function scrolls through the reviews to load more content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def scroll_reviews_panel(page, max_scrolls=10, max_reviews=20):\n",
    "    \"\"\"Scrolls through reviews panel to load all reviews.\"\"\"\n",
    "    scrollable_div = page.locator('div[role=\"main\"] div[jslog$=\"mutable:true;\"]').first\n",
    "    previous_review_count = 0\n",
    "    scroll_attempts = 0\n",
    "    no_change_count = 0\n",
    "    \n",
    "    print(\"📜 Scrolling to load reviews...\")\n",
    "    \n",
    "    while scroll_attempts < max_scrolls:\n",
    "        review_elements = page.locator(\"div[data-review-id][jsaction]\")\n",
    "        current_review_count = await review_elements.count()\n",
    "        \n",
    "        if current_review_count >= max_reviews:\n",
    "            break\n",
    "        \n",
    "        print(f\"   Scroll {scroll_attempts + 1}/{max_scrolls}: Found {current_review_count} reviews\")\n",
    "        \n",
    "        await scrollable_div.evaluate(\"\"\"\n",
    "            (element) => {\n",
    "                element.scrollTo(0, element.scrollHeight + 100);\n",
    "            }\n",
    "        \"\"\")\n",
    "        \n",
    "        await asyncio.sleep(2)\n",
    "        \n",
    "        if current_review_count == previous_review_count:\n",
    "            no_change_count += 1\n",
    "            if no_change_count >= 3:\n",
    "                print(f\"   No new reviews loaded. Finished.\")\n",
    "                break\n",
    "        else:\n",
    "            no_change_count = 0\n",
    "        \n",
    "        previous_review_count = current_review_count\n",
    "        scroll_attempts += 1\n",
    "    \n",
    "    final_count = await review_elements.count()\n",
    "    print(f\"✅ Loaded {final_count} reviews!\\n\")\n",
    "    return final_count"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Scrape Google Reviews\n",
    "Main scraping function using Playwright."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async def scrape_google_reviews(url, max_reviews=20):\n",
    "    \"\"\"Scrape reviews from Google Maps URL.\"\"\"\n",
    "    reviews = []\n",
    "    \n",
    "    async with async_playwright() as p:\n",
    "        print(\"🌐 Opening Google Maps...\")\n",
    "        browser = await p.chromium.launch(headless=True)\n",
    "        context = await browser.new_context()\n",
    "        page = await context.new_page()\n",
    "        \n",
    "        await page.goto(url)\n",
    "        \n",
    "        print(\"⏳ Waiting for reviews to load...\")\n",
    "        review_html_elements = page.locator(\"div[data-review-id][jsaction]\")\n",
    "        \n",
    "        try:\n",
    "            await review_html_elements.first.wait_for(state=\"visible\", timeout=10000)\n",
    "        except:\n",
    "            print(\"❌ Could not find reviews on this page.\")\n",
    "            await browser.close()\n",
    "            return []\n",
    "        \n",
    "        # Scroll to load more reviews\n",
    "        total_reviews = await scroll_reviews_panel(page, max_scrolls=10, max_reviews=max_reviews)\n",
    "        \n",
    "        print(\"📥 Extracting review data...\")\n",
    "        review_html_elements = page.locator(\"div[data-review-id][jsaction]\")\n",
    "        all_reviews = await review_html_elements.all()\n",
    "        \n",
    "        for idx, review_html_element in enumerate(all_reviews, 1):\n",
    "            try:\n",
    "                # Extract rating\n",
    "                stars_element = review_html_element.locator(\"[aria-label*=\\\"star\\\"]\")\n",
    "                stars_label = await stars_element.get_attribute(\"aria-label\")\n",
    "                \n",
    "                stars = None\n",
    "                for i in range(1, 6):\n",
    "                    if stars_label and str(i) in stars_label:\n",
    "                        stars = i\n",
    "                        break\n",
    "                \n",
    "                # Expand \"More\" button if present\n",
    "                more_element = review_html_element.locator(\"button[aria-label=\\\"See more\\\"]\").first\n",
    "                if await more_element.is_visible():\n",
    "                    await more_element.click()\n",
    "                    await asyncio.sleep(0.3)\n",
    "                \n",
    "                # Extract review text\n",
    "                text_element = review_html_element.locator(\"div[tabindex=\\\"-1\\\"][id][lang]\")\n",
    "                text = await text_element.text_content()\n",
    "                \n",
    "                reviews.append({\n",
    "                    'rating': stars,\n",
    "                    'text': text.strip()\n",
    "                })\n",
    "                \n",
    "                if idx % 5 == 0:\n",
    "                    print(f\"   Extracted {idx}/{total_reviews} reviews...\")\n",
    "            \n",
    "            except Exception as e:\n",
    "                continue\n",
    "        \n",
    "        await browser.close()\n",
    "        \n",
    "        print(f\"✅ Successfully extracted {len(reviews)} reviews!\\n\")\n",
    "        return reviews"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Analyze Reviews with LLM\n",
    "Uses Llama 3.2 via Ollama to analyze sentiment and extract insights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def analyze_reviews(reviews, model=\"llama3.2\"):\n",
    "    \"\"\"Analyze and summarize reviews using Llama 3.2.\"\"\"\n",
    "    \n",
    "    if not reviews:\n",
    "        return \"No reviews to analyze.\"\n",
    "    \n",
    "    # Prepare reviews text\n",
    "    reviews_text = \"\\n\\n\".join([\n",
    "        f\"Rating: {r['rating']}/5\\n{r['text']}\" \n",
    "        for r in reviews\n",
    "    ])\n",
    "    \n",
    "    # Limit text length\n",
    "    reviews_text = reviews_text[:8000]\n",
    "    \n",
    "    prompt = f\"\"\"Analyze the following Google Maps reviews and provide:\n",
    "\n",
    "1. **Overall Sentiment**: Positive, Negative, or Mixed\n",
    "2. **Key Themes**: Main topics mentioned\n",
    "3. **Pros**: What customers love (3-5 bullet points)\n",
    "4. **Cons**: What customers dislike (3-5 bullet points)\n",
    "5. **Summary**: A 2-3 sentence overall summary\n",
    "\n",
    "REVIEWS:\n",
    "{reviews_text}\n",
    "\"\"\"\n",
    "    \n",
    "    print(\"🧠 Analyzing reviews with Llama 3.2...\")\n",
    "    print(\"   (This may take a moment...)\\n\")\n",
    "    \n",
    "    try:\n",
    "        response = ollama.chat(\n",
    "            model=model,\n",
    "            messages=[\n",
    "                {\n",
    "                    \"role\": \"system\",\n",
    "                    \"content\": \"You are an expert at analyzing customer reviews and extracting insights.\"\n",
    "                },\n",
    "                {\n",
    "                    \"role\": \"user\",\n",
    "                    \"content\": prompt\n",
    "                }\n",
    "            ]\n",
    "        )\n",
    "        return response[\"message\"][\"content\"]\n",
    "    \n",
    "    except Exception as e:\n",
    "        return f\"❌ Analysis error: {e}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Usage\n",
    "\n",
    "### Test with Sultanahmet Mosque (Blue Mosque) in Istanbul"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example URL - Sultanahmet Mosque\n",
    "url = \"https://www.google.com/maps/place/Sultanahmet+Camii/@41.0054096,28.9768138,17z\"\n",
    "\n",
    "# Number of reviews to scrape\n",
    "max_reviews = 15\n",
    "\n",
    "# Scrape reviews\n",
    "reviews = await scrape_google_reviews(url, max_reviews=max_reviews)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Display Sample Reviews"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"\\n📝 Sample Reviews ({min(3, len(reviews))} of {len(reviews)}):\")\n",
    "print(\"=\" * 60)\n",
    "\n",
    "for i, review in enumerate(reviews[:3], 1):\n",
    "    rating_stars = \"⭐\" * (review['rating'] or 0)\n",
    "    print(f\"\\n{i}. {rating_stars}\")\n",
    "    print(f\"   {review['text'][:200]}...\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Analyze Reviews with Llama 3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Analyze the reviews\n",
    "analysis = analyze_reviews(reviews)\n",
    "\n",
    "# Display as formatted markdown\n",
    "display(Markdown(f\"## 📊 Review Analysis\\n\\n{analysis}\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Try Your Own Location!\n",
    "\n",
    "Replace the URL below with any Google Maps place:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your custom URL\n",
    "custom_url = \"https://www.google.com/maps/place/YOUR_PLACE_HERE\"\n",
    "\n",
    "# Scrape and analyze\n",
    "custom_reviews = await scrape_google_reviews(custom_url, max_reviews=20)\n",
    "custom_analysis = analyze_reviews(custom_reviews)\n",
    "\n",
    "display(Markdown(f\"## 📊 Analysis\\n\\n{custom_analysis}\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What I Learned\n",
    "\n",
    "- **Web Scraping**: Using Playwright for dynamic content\n",
    "- **Async Programming**: Handling asynchronous operations in Python\n",
    "- **DOM Navigation**: Finding and extracting specific HTML elements\n",
    "- **LLM Integration**: Using Llama 3.2 for sentiment analysis\n",
    "- **Prompt Engineering**: Crafting effective analysis prompts\n",
    "- **Data Processing**: Structuring and cleaning scraped data"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
